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Letter from the Editor 


AIR is pleased to reintroduce Professional Files. This volume contains three articles recom- 
mended for inclusion by AIR members through a blind peer-review process. Each article 
puts forth ideas and analysis for our consideration as institutional research professionals. 
Most salient for me in my role as volunteer coordinating editor is that the articles frame 
the work of IR as more than pure technical analysis through acknowledgment of the im- 
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portance of context. 

Zan, Yoon, Khasawneh, and Srihari unpack differences in enrollment projections, with a particular focus on the influ- 
ence of the unit of analysis on the appropriateness of the model. Saupe and Eimers examine the under-appreciated 
area of grade point average reliability; something that will be of greater importance as policy dialogues move to multi- 
ple measures of college readiness and adjusted metrics of student outcomes. Crisp, Palacios, and Kaulfus empower IR 
professions in efforts to better understand student engagement efforts on campus. 

It has been an honor to be a part of this volume of Professional Files and to strive to elevate the work of peers for the 
benefit of the field. We look forward to receiving your submission next. 


Sincerely, 



Christopher M. Mullin 
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Abstract 

The following article describes pro- 
grams used by universities and colleges 
to engage students; these programs 
include mentoring, learning communi- 
ties, and first-year success courses and 
programs. We begin with a brief over- 
view of student development theory, 
program descriptions and citations, 
and article summaries for key refer- 
ences. Next, we introduce prominent 
national surveys available to institu- 
tions that are interested in measuring 
student engagement (inside or outside 
formal programs). We conclude with 
additional references and recommen- 
dations for institutional researchers 
involved in program review and/or stu- 
dent outcomes assessment of student 
engagement programs. 

INTRODUCTION 

Higher education is not a passive expe- 
rience that leaves students untouched. 
Rather, college life involves a variety of 
experiences, both inside and outside 
the classroom, designed to engage 


students and enhance their lives by 
introducing new ideas, challenging 
past behaviors or events, and creat- 
ing intellectual discord and tension 
(Keeling, Wall, Underhile, & Dungy, 
2008). Institutional effectiveness is 
dependent, in part, on institutions 
providing students with opportuni- 
ties to purposefully engage (Harper & 
Quaye, 2009). According to Pascarella, 
"an excellent undergraduate education 
is most likely to occur at those colleges 
and universities that maximize good 
practices and enhance student engage- 
ment" (2001 , p. 22). As such, institutions 
that value student success will take 
every opportunity to engage students 
both academically and socially (Culp, 
2007). 

Simply defined, student engagement is 
how universities organize their human 
capital and resources to encourage 
students to involve themselves in aca- 
demic, interpersonal, and cocurricular 
activities (Astin, 1 993). Student engage- 
ment is typically not viewed as a direct 
measure of student learning, but rather 
is used as a measurement of participa- 
tion in meaningful educational experi- 
ences and activities that facilitate both 
social and academic integration (Tinto, 
2000) and lead to student development 
(LaNasa, Olson, & Alleman, 2007). More 
specifically, opportunities for students 
to engage are provided through for- 
malized programs designed to directly 
support student integration and/or de- 
velopment outcomes (i.e., study strate- 
gies, time/stress management skills, 
motivation, academic self-confidence, 


connections with peers, and out-of- 
class interactions with faculty), that in 
turn directly impact traditional mea- 
sures of student success (i.e., grades, 
persistence). 

According to a survey involving 1 85 
colleges and universities across the 
country, the most prevalent services 
and programs provided to students to 
promote student engagement in the 
first year include tutoring, academic 
coaching and counseling, writing sup- 
port services, academic advising, and 
testing services (National Resource 
Center for the First-Year Experience 
and Students in Transition, 2008). In 
large part due to work by Kuh and col- 
leagues, engagement programs and 
activities have become increasingly 
viewed as an important component 
of student success (Kuh, Kinzie, Buck- 
ley, Bridges, & Hayek, 2007). As such, 
increasing attention has been given to 
the implementation, administration, 
and assessment of educational experi- 
ences designed to engage students. 

Although engagement programs are 
typically created and managed by stu- 
dent affairs professionals, institutional 
researchers should be familiar with 
programmatic efforts on their campus, 
and should understand how program 
outcomes can be used to address ac- 
creditation standards and institutional 
planning and assessment goals (as 
demonstrated in volume 141 of New 
Directions for Institutional Research, 
2009). The present article describes 
several programs currently used by 
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Table 1 . Student Engagement Theory References 


References for Student 
Engagement Theory 

Impact for Institutional 

Researchers 

Astin, A. (1 993). What matters in col- 
lege: Four critical years revisited. San 
Francisco: Jossey-Bass. 

Presents the results from a study of 
almost 25,000 students at 200 colleges 
and universities. Findings demonstrate 
how colleges and universities can 
enhance student development during 
college through a variety of in-class 
and out-of-class experiences. 

Hamrick, F. A., Evans, N. J., & Schuh, 

J. H. (2002). Foundations of student 
affairs practice: How philosophy, theory, 
and research strengthen educational 
outcomes. San Francisco: Jossey- 
Bass. 

Provides an overview of theories that 
relate to student growth and develop- 
ment during college and explores ways 
that institutions can enhance students’ 
educational experiences. 

Harper, S. R., & Quaye, S. J. (2009). 
Student engagement in higher edu- 
cation: Theoretical perspectives and 
practical approaches for diverse popu- 
lations. New York: Routledge. 

Based on theory, explores ways that 
diverse populations of students (e.g., 
racial and ethnic minorities, LGBT 
students) as compared to nondiverse 
populations might struggle to engage 
during college. 

Kuh, G. D„ Kinzie, J., Schuh, J. H„ & 
Whitt, E. J. (2005). Assessing condi- 
tions to enhance educational effective- 
ness. The inventory for student en- 
gagement and success. San Francisco: 
Jossey-Bass. 

Provides a theoretical framework, the 
Inventory for Student Engagement and 
Success (ISES), to examine student 
engagement within a program, division, 
college, or entire institution. Explains 
how information can be used for pro- 
gram reviews, planning, and accredita- 
tion. 

Pascarella, E., & Terenzini, P. T. (2005). 
How college affects students: A third 
decade of research. San Francisco: 
Jossey-Bass. 

Presents findings from 15 years of 
research on college’s impact on student 
learning, growth, and development. 

Also presents implications for research, 
policy, and practice. 

Tinto, V. (1993). Leaving college: 
Rethinking the causes and cures of 
student attrition (2nd ed.). Chicago: 
University of Chicago Press. 

Synthesizes research on student reten- 
tion demonstrating the importance of 
institutions providing students with op- 
portunities to engage with the campus 
community. 


postsecondary institutions to engage 
students with the intent of providing 
institutional researchers with knowl- 
edge to support assessment efforts. 

The article begins with an overview of 
relevant student development theory 
that serves as a conceptual ground- 
ing for engagement programs. Next, 
we provide program descriptions for 
programs that have been linked to 
engagement (e.g., social engagement, 
academic skills, time management, 
and career selection) and academic 
outcomes. Citations and article sum- 
maries for key references are provided 
in table form following each section for 
institutional research professionals who 
are interested in learning more about 
student engagement and/or enrich- 
ment programs. Third, we highlight 
prominent national surveys available to 
institutions that are interested in mea- 
suring student engagement (inside or 
outside of formal programs). The article 
concludes with additional references 
and recommendations for institutional 
researchers involved in program review 
and/or student outcomes assessment 
of student engagement programs. 

OVERVIEW 
OF STUDENT 
DEVELOPMENT 
THEORY 

Research and theory by Erikson and 
Chickering provide a foundation for our 
current understanding of student de- 
velopment. Erikson's theory of psycho- 
social development (1 968) explains that 
individuals must work through eight 
stages in order to successfully form 
an identity and discover purpose and 
meaning in life. According to Erikson, 
adolescents move through a develop- 
mental stage during college termed the 
"identity versus role confusion" stage 
before moving into adulthood. This 
stage involves students successfully, or 
in some cases unsuccessfully, develop- 


ing a personal identity; it is defined by 
a "crisis" that must be resolved in order 
for students to avoid an "identity crisis" 
that leads to stagnation or regression. 
Similarly, Chickering's seven vectors of 


student development (1 969) explain 
that college students move through 
seven vectors or stages as they become 
more self-aware and as they have more 
complex thoughts, which is spurred 
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by interactions with peers and faculty 
and the introduction of new concepts 
and ideas. Chickering's work has since 
been updated to be inclusive of non- 
traditional students (i.e., Chickering & 
Reisser, 1993). 

More recently, Erikson and Chickering's 
work has been expanded by Astin and 
Tinto in an effort to understand the 
factors related to student success and 
persistence. Astin's theory of involve- 
ment (1 984, 1 999) postulates that 
student involvement in college has a 
direct impact on psychosocial develop- 
ment and assists in identity formation 
as students work toward graduation. 
Astin's work also demonstrates that 
student learning and development are 
dependent on active involvement in 
academic and social aspects of a col- 
lege experience. Moreover, his theory 
argues that development is influenced 
by both the quality and the quantity of 
involvement. 

Similarly, Tinto's theory of student de- 
parture (1 993) demonstrates that stu- 
dents are more likely to persist toward 
graduation if they become socially and 
academically integrated into the col- 
lege environment. He postulates that 
integration is achieved when a student 
and the institution share similar values 
and the student is engaged in positive 
social and academic interactions. Tinto's 
work demonstrates the importance 
of support from faculty and university 
staff. Table 1 contains key references 
related to theory underpinning student 
engagement programs to guide the 
development of programmatic activi- 
ties and goals. 

ENGAGEMENT 

PROGRAMS 

The following section provides a 
program overview for mentoring, 
learning communities, and first-year 
success courses and programs. This is 
not meant to provide a comprehensive 


overview of engagement programs, 
but rather to provide institutional 
researchers with examples of programs 
currently employed on college and 
university campuses across the country 
that have been empirically shown to 
enhance students' experiences and to 
promote students engagement. 

Mentoring Programs 

Mentoring programs that involve a vari- 
ety of engagement activities such as 
academic advising, academic skills de- 
velopment, personal development, and 
career selection are becoming increas- 
ingly prevalent. Mentoring programs 
and experiences have been empirically 
shown to be associated with numer- 
ous academic and developmental 
outcomes, including improving critical 
thinking skills, self-confidence, per- 
sistence, and academic performance. 
Mentoring has also been found to help 
students develop their latent abilities, 
and to raise students' expectations and 
future aspirations (e.g., Astin, 1 999; 
Bank, Slavings, & Biddle, 1990; Camp- 
bell & Campbell, 1 997; Freeman, 1 999; 
Girves, Zepeda, & Gwathmey, 2005; 
Johnson, 1989; Mangold, 2003; Pagan 
& Edwards-Wilson, 2003; Roberts, 2000; 
Ross-Thomas & Bryant, 1 994). 

Institutional researchers should con- 
sider and draw from published pro- 
gram overviews and evaluations when 
assisting in the development and/or 
assessment of programmatic efforts. 
Unfortunately, there is little agreement 
regarding how college students experi- 
ence mentoring, or on the components 
that should be included in a mentoring 
program. Moreover, it has been noted 
that the majority of empirical work on 
mentoring has been limited due to 
methodological weaknesses including 
limitations in how mentoring is defined 
and measured, a lack of sophisticated 
data analysis and theoretical ground- 
ing, failure to control for selection bias, 
and an overreliance of self-reported 
benefits of mentoring as the assess- 


ment measure (Crisp & Cruz, 2009). 
However, according to a comprehen- 
sive review of the psychological, busi- 
ness, and education literature by Nora 
and Crisp (2007), students perceive 
a holistic mentoring experience to 
include four separate yet interrelated 
types of support: (1) psychological 
and emotional support, (2) support for 
setting goals and choosing a career 
path, (3) academic subject knowledge 
support aimed at advancing a student's 
knowledge relevant to his or her cho- 
sen field, and (4) support in the form of 
a role model. 

An assortment of mentoring programs 
designed to serve a variety of student 
populations including first-generation, 
minority, at-risk, and/or low-income 
students have been described in the 
literature (e.g., Bordes & Arredondo, 
2005; Pagan & Edwards-Wilson, 2003; 
Wallace, Abel, & Ropers-Huilman, 

2000). For instance, the Puente Project, 
evaluated by Laden (1999), is a nation- 
ally recognized program designed to 
raise Latino/a students' educational 
and career aspirations. Other examples 
of programs that involve a mentoring 
component includeTRIO Programs 
(Wallace et al., 2000), the Adventor Pro- 
gram (Shultz, Colton, & Colton, 2001), 
and the Search for Education, Eleva- 
tion and Knowledge (SEEK) Program 
(Sorrentino, 2007). Table 2 provides a 
list of select published work including a 
mentoring theory and scale (i.e., Crisp, 
2009) to guide assessment efforts. 

Learning Communities 

Recently, there has been increased 
interest from both academic and 
student affairs practitioners to enhance 
and/or expand innovative programs 
such as learning communities and 
first-year experiences (Dale & Drake, 
2007). Learning communities provide 
college students with the opportunity 
to get to know other students as well 
as faculty; these communities integrate 
students into the university commu- 
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Table 2. Mentoring Program References 


References for Designing/ 
Assessing Outcomes for 

Mentoring Programs 

Impact for Institutional 

Researchers 

Crisp, G. (2009). Conceptualization and 
initial validation of the College Student 
Mentoring Scale (CSMS). Journal of 
College Student Development, 50(2), 
177-194. 

Offers a theoretically grounded survey 
to be used by institutions that are 
interested in measuring the mentoring 
experiences of undergraduate college 
students. Includes the 25-item survey 
as an appendix. 

Laden, B. V. (1999). Socializing and 
mentoring college students of color: 

The Puente Project as an exem- 
plary celebratory socialization model. 
Peabody Journal of Education, 74(2), 
55-74. 

Presents highlights from the Puente 
Project, a program designed to support 
first-generation Latino/a college stu- 
dents at California community colleges. 

Shultz, E. L., Colton, G. M., & Colton, 

C. (2001). The Adventor Program: 
Advisement and mentoring for students 
of color in higher education. Journal of 
Humanistic Counseling, Education, and 
Development, 40, 208-218. 

Describes the Adventor Program, 
designed by the College of Education 
at Kutztown University to serve racial/ 
minority students. Presents informa- 
tion about the program design and 
implementation, and follows with pilot 
findings. 

Sorrentino, D. M. (2007). The SEEK 
mentoring program: An application of 
the goal-setting theory. Journal of Col- 
lege Student Retention, 8(2), 241-250. 

Presents a description and overview 
of the SEEK Program at the College of 
Staten Island, City University of New 

York (CUNY), that provides academic 
mentoring to students at risk for aca- 
demic dismissal. 

Wallace, D., Abel, R., & Ropers- 
Huilman, B. R. (2000). Clearing a path 
for success: Deconstructing borders 
through undergraduate mentoring. The 
Review of Higher Education, 24(1 ), 
87-102. 

Utilizes qualitative interview data to 
explore first-generation, low-income 
students’ experiences with and percep- 
tions of formalized mentoring programs. 


nity in a meaningful way (Price, 2005; 
Taylor, Moore, MacGregor, & Lindblad, 
2003; Tinto, 1 998; Weber, 2000). More 
specifically, learning communities 
provide students with an educational 
environment that supports student 
engagement through an integrated 
and interdisciplinary curriculum that 
may cross departments or divisions and 
focuses on high levels of participation 
and support from faculty (Brower & 
Dettinger, 1998; Oertel as cited in Taylor 


et al., 2003; Price, 2005). Learning com- 
munities exist in a variety of formats 
to facilitate students' connection to 
faculty, other students, and the institu- 
tion (Tinto, 1998); these formats include 
team-taught programs, paired or 
clustered courses, cohorts of students 
enrolled together in large courses, and 
residence-based programs (Price, 2005; 
Taylor et al., 2003; Tinto, 1 998; Weber, 
2000 ). 


There is a wealth of literature on learn- 
ing communities to suggest that pro- 
grammatic efforts can be used to influ- 
ence retention and learning outcomes. 
Namely, ongoing evaluations of the 
Opening Doors Learning Communities 
(ODLC) program by MDRC are utilizing 
experiments that test a cause-and-ef- 
fect relationship between participation 
in learning communities and outcomes 
for various groups of students through 
the use of random assignments (e.g., 
Bloom & Sommo, 2005; Richburg- 
Hayes, Visher, & Bloom, 2008; Scrivener, 
Bloom, LeBlanc, Paxson, & Sommo, 
2008). Additionally, nonexperimental 
research by Zhao and Kuh (2004) has 
revealed that students who partici- 
pate in the learning community have 
higher levels of academic effort, active 
learning, interactions with faculty, and 
participation in diversity activities. 
Participants also reported more posi- 
tive associations with advisers, campus 
support services, and overall experi- 
ences, as well as self-reported gains in 
personal and social development and 
basic skills advancement. Furthermore, 
qualitative work by Tinto and Goodsell 
(1993) involving a linked writing course 
and seminar found that learning com- 
munities supported the development 
of students' time management, writing, 
and study skills. Table 3 provides a list 
of key references to studies on learning 
communities. 

First-Year/Orientation/Success 

Programs 

According to a survey by the National 
Resource Center for the First-Year 
Experience and Students in Transition 
(2008), nearly 85% of colleges and 
universities currently offer a first-year 
program. First-year programs, stu- 
dent success courses, and orientation 
courses all focus on assisting college 
students' transition and/or enhancing 
engagement and success in college 
(Cook, 1 996). These programs are de- 
signed to teach students strategies for 
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Table 3. Learning Communities References 


References for Designing/ 
Assessing Outcomes for Learn- 
ing Communities 

Impact for Institutional 

Researchers 

Richburg-Hayes, L., Visher, M. G., & 
Bloom, D. (2008). Do learning com- 
munities affect academic outcomes? 
Evidence from an experiment in a com- 
munity college. Journal of Research on 
Educational Effectiveness, 1( 1), 33-65. 

Details the design and results of a 
study that utilized an experimental 
design to explore a cause-and-effect 
relationship between participation in 
a learning community and students’ 
engagement and attachment to the 
institution’s community. 

Scrivener, S., Bloom, D., LeBlanc, A., 
Paxson, C. E., & Sommo, C. (2008). 

A good start: Two-year effects of a 
freshmen learning community program 
at Kingsborough Community College. 
MDRC, New York. 

Explains how Kingsborough’s Opening 
Doors Learning Communities (ODLC) 
program utilizes an experimental de- 
sign with random assignment to study 
the effect of learning communities on 
student success. 

Taylor, K., Moore, W. S., MacGregor, 

J., Lindblad, J. (2003). What we know 
now. National Learning Communities 
Project Monograph Series. The Wash- 
ington Center for Improving the Quality 
of Undergraduate Education at The 
Evergreen State College in coopera- 
tion with the American Association for 
Higher Education, Washington, DC. 

Presents findings from a systematic 
literature review of research and as- 
sessment specific to learning communi- 
ties conducted by the National Learning 
Communities Project. 

Tinto, V. (1997). Classrooms as 
communities: Exploring the educa- 
tional character of student persistence. 
Journal of Higher Education, 68(6), 
599-623. 

Presents a mixed methods study of the 
use of learning communities at Seattle 
Central Community College. The study 
investigates the impact on collabora- 
tive learning strategies on both student 
persistence and learning. 

Tinto, V. (1998). Learning communities 
and the reconstruction of remedial edu- 
cation in higher education. Prepared 
for presentation at the “Conference on 
Replacing Remediation in Higher Edu- 
cation” at Stanford University, January 
26-67, 1998, sponsored by the Ford 
Foundation at the U.S. Department of 
Education. 

Details a case study specific to the 
effectiveness of a developmental edu- 
cation learning community. Provides a 
description of the program, research 
procedures, and findings. 

success in college by introducing them 
to campus facilities, resources, and 
services; and/or by enhancing students' 
health or well-being, study skills, time 
management, or learning styles (e.g., 
Derby, 2007; Derby & Smith, 2004; 

Derby & Watson, 2006; Glass & Garrett, 
1995; Grunder& Hellmich, 1996; Na- 
tional Resource Center for the First-Year 
Experience and Students in Transition, 
2008; O'Gara, Karp, & Hughes, 2008; 
Raymond & Napoli, 1 998). Programmat- 


ic efforts may be offered as both credit 
and noncredit courses (Donnangelo & 
Santa Rita, 1 982), and may be required 
or optional (Zimmerman, 2000). Pos- 
sible program offerings range from 
a half-day orientation (Hollins, 2009) 
to semester- or year-long programs 
(Donnangelo & Santa Rita, 1 982; Glass 
& Garrett, 1995). 

Although the majority of research to 
date has focused on examining the 
impact of programs on retention or 
learning outcomes (e.g., Derby & Smith, 
2004; Glass & Garrett, 1 995; Pascarella, 
Terenzini, & Wolfle, 1986; Raymond & 
Napoli, 1 998; Stovall, 1 999), findings 
from the National Resource Center for 
the First-Year Experience and Students 
in Transition (2008) survey indicate 
that success courses may also be as- 
sociated with engagement outcomes 
such as increasing peer connections, 
use of campus services, participation 
in campus services, and out-of-class 
interaction with faculty. Additionally, 
research conducted by the Commu- 
nity College Research Center (CCRC) at 
Teachers College, Columbia University, 
found that programmatic efforts may 
integrate students both socially and 
academically by helping to facilitate 
the development of students' relation- 
ships with faculty and other students 
(O'Gara et al., 2008). Moreover, evalua- 
tion efforts at the Virginia Community 
College System examined the impact of 
a comprehensive approach to student 
orientation that included a half- to 
full-day program (Seeking Oppor- 
tunities through Academic Recruit- 
ment (SOAR]), group advising, and an 
orientation course. Findings indicated 
that the program increased students' 
personal adjustment during the 
transition process and academic gains 
among first-semester students. The 
orientation course was also found to 
assist students in developing effective 
study habits, career and academic plan- 
ning, and knowledge regarding college 
resources (Hollins, 2009). 
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Table 4. First-Year/Orientation/Success Program References 


References for Designing/ 
Assessing Outcomes for First 
Year/Orientation Success 

Programs 

Impact for Institutional 

Researchers 

Engberg, M. E., & Mayhew, M. J. 

(2007). The influence of first year 
“success” courses on student learning 
and democratic outcomes. Journal of 
College Student Development, 49(2), 
95-109. 

Examines the impact of a first-year 
program on a variety of outcomes, 
including multicultural awareness, 
commitment to social justice, and the 
complexity of attributes. Also includes 
a discussion of connecting theory with 
practice. 

Keup, J. Ft., & Barefoot, B. O. (2005). 
Learning how to be a successful stu- 
dent: Exploring the impact of first year 
seminars on student outcomes. Journal 
of the First year Experience, 17 ( 1 ), 

11-47. 

Utilizes data from the Cooperative In- 
stitutional Research Program’s (CIRP) 
2000 freshman survey and the Your 

First College Year (YFCY) 2001 survey 
to study the relationship between taking 
a first-year course and academic and 
social experiences of students. 

O’Gara, L. Karp, M. M., & Hughes, K. 

L. (2009). Student success courses in 
the community college: An exploratory 
study of student perspectives. Commu- 
nity College Review, 36(3), 195-218. 

Investigates the perceived impact of 
taking a success course on individual 
benefits such as building relationships 
with faculty and developing study skills 
at two urban community colleges in the 
Northeast. 

Schwitzer, A. M., McGovern, T. V., & 
Robbins, S. B. (1991). Adjustment 
outcomes of a freshman seminar: A 
utilization-focused approach. Journal 
of College Student Development, 321, 
484-489. 

Presents findings of an evaluation 
investigating the relationship between 
participation in a college orientation 
seminar and students’ social and aca- 
demic adjustment in college. 

Zeidenberg, M., Jenkins, D., & Cal- 
cagno, J. C. (2007). Do student suc- 
cess courses actually help community 
college students succeed? Community 
College Research Center (CCRC) Brief 
No. 36, June. 

Examines the impact of enrolling in 
a student success course over the 
course of 17 semesters on various stu- 
dent outcomes, controlling for possible 
extraneous variables. 


Resources and support regarding 
first-year programs are available to 
researchers through the First-Year 
Experience (http://www.sc.edu/fye/). 
Additionally, Table 4 provides a list of 
references specific to designing and as- 
sessing outcomes for first-year, orienta- 
tion, and student success courses and 
programs. 

TOOLS FOR 
ASSESSING STUDENT 
ENGAGEMENT 

Several national surveys are available 
to institutions interested in assessing 
student engagement and/or students' 
experiences during college, including 
the National Survey of Student Engage- 
ment (NSSE; http://nsse.iub.edu/). This 
survey contains items assumed to mea- 
sure different components of student 
engagement, including academic chal- 
lenge (e.g., preparing for class, using 
higher-order thinking skills), active and 
collaborative learning (e.g., contribut- 
ing to class discussions, working with 
students outside of class), and student 
interactions with faculty members (e.g., 
talking about career plans, working 
on activities other than coursework) 
(Kuh, 2004). Additionally, seniors report 
whether they participated in various 
programs and on-campus activities, 
including learning communities. The 
NSSE is typically administered in the 
spring using a paper or online version 
of the survey to a random sample of 
first-year and senior students (Kuh, 
Cruce, Shoup, Kinzie, & Gonyea, 2008). 

Variations of the NSSE that measure 
engagement of different student 
populations are also available, includ- 
ing the Community College Survey of 
Student Engagement (CCSSE) (http:// 
www.ccsse.org/) and Beginning College 
Survey of Student Engagement (BCSSE) 
(http://bcsse.iub.edu/). Moreover, many 
institutions compare student responses 
from the NSSE with faculty perceptions 


measured by the Faculty Survey of Stu- 
dent Engagement (FSSE) (http://fsse. 
iub.edu/). Tips and recommendations 
for analyzing and interpreting the NSSE 
survey data are available in a 2009 is- 
sue of New Directions for Institutional 
Research by Chen and colleagues. 
Another survey available to institutions 
interested in assessing students' devel- 


opment during the first year of college 
is the Your First College Year (YFCY) 
survey, developed through collabora- 
tion between the Higher Education 
Research Institute (FIERI) and the Policy 
Center on the First Year of College at 
Brevard College (http://www.heri.ucla. 
edu/yfcyoverview.php). This survey 
allows colleges and universities to 
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identify students' experiences during 
the first year that encourage and sup- 
port student involvement, satisfaction, 
and learning, as well as other success 
indicators that enhance first-year 
programs. Similar to the NSSE, the YFCY 
allows for comparisons to national and 
institutional peer groups among par- 
ticipating institutions as well as trend 
and longitudinal analyses. The YFCY is 
offered in both paper and web format 
and is conducted at the end of the stu- 
dents' first academic year (somewhere 
between the months of March to June). 

Third, the Degrees of Preparation sur- 
vey may also be of interest to institu- 
tions in measuring ways that college 
experiences are related to various 
developmental and civic outcomes, 
including critical thinking skills, career- 
related experiences, and civic engage- 
ment. This survey's major components 
and question descriptions are avail- 
able in an issue of New Directions for 
Institutional Research (Ouimet & Pike, 
2008). A copy of the piloted version of 
the survey is available at http://www. 
aascu.org/accountability/survey/?u=l. 
Additional information regarding the 
above-mentioned instruments as well 
as an inventory of other potentially 
relevant surveys and tools used to as- 
sess student engagement outcomes is 
posted on the Association for Institu- 
tional Research (AIR) website at http:// 
applications.airweb.org/surveys/De- 
fault.aspx. 

CONCLUSIONS AND 
RECOMMENDATIONS 

We hope that the information present- 
ed in this article is useful to institutional 
researchers involved with program 
planning, assessment, and/or accredi- 
tation efforts; we offer the following 
conclusions and recommendations. 
First, we recommend that student 
engagement programs be clearly con- 
nected to the institution's core mission 
(Culp, 2007) and that they be grounded 


in student development theory (Dale 
& Drake, 2007). Institutional research- 
ers should work with faculty as well as 
academic and student affairs personnel 
to utilize previously validated assess- 
ment tools and survey items that 
are grounded in theory, rather than 
developing home-grown surveys that 
may or may not be accurate measures 
of students' experiences. Additional 
recommendations specific to using 
engagement data in assessment and 
planning efforts are provided by Banta, 
Pike, and Hansen (2009). 

When possible, we also strongly 
encourage the use of experimental 
designs that utilize random assignment 
to groups and an experimental (i.e., 
students in the program) and control 
group (i.e., group of students who do 
not participate) to assess cause-and- 
effect relationships between program 
activities and engagement outcomes. 
Examples of evaluation work utilizing 
experimental designs are provided in 
the learning community section of this 
article. Because experimental designs 
are rarely possible, we also recommend 
the use of quasi-experimental designs 
that adequately control for possible 
confounding variables (e.g., matching 
groups). Furthermore, in cases where 
the program is already in place or the 
independent variable (i.e., program) 
cannot be manipulated, we suggest 
the use of nonexperimental designs 
that adequately control for students' 
background characteristics and precol- 
lege characteristics that have been 
previously found to impact student 
outcomes (see discussion by Cole, 
Kennedy, & Ben-Avie, 2009). Finally, we 
suggest that institutional researchers 
consider using qualitative methods 
to answer "how" and "why" questions 
specific to program assessment. 

Next, we encourage institutional re- 
searchers to actively seek out collabo- 
rations with faculty and student and 
academic affairs programs/offices. Stu- 


dent affairs personnel have knowledge 
of long-standing and professionally 
accepted student development theory 
(King & Howard-Hamilton, 2000) that is 
likely to strengthen assessment efforts 
and properly take into account factors 
such as ethnicity, culture, and career 
choice (Pickering & Sharpe, 2000). Stu- 
dent and academic affairs practitioners 
and faculty also have ready access and 
can encourage student participation 
in surveys needed to properly assess 
student outcomes (Smith & Mather, 
2000). Moreover, institutional research 
offices may be able to provide precol- 
lege data to student affairs divisions 
to guide and inform the development 
of programmatic activities (Cole et al., 
2009). Kinzie and Pennipede (2009) 
provide further discussion and recom- 
mendations for collaborating with 
student affairs in using data. Addition- 
ally, a New Directions article by Nelson 
Laird, Smallwood, Niskode-Dossett, and 
Garver (2009) offers ideas for involving 
faculty in assessment efforts specific to 
student engagement. 
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